Bootstrapping Techniques in Statistical Analysis and Approaches in R MATH

The true probability distribution of a test statistic is rarely known. Generally, its asymptotic law is used as approximation of the true law. If the sample size is not large enough, the asymptotic behavior of that statistic could lead to a poor approximation of the true one. Using bootstrap methods, under some regularity conditions, it is possible to obtain a more accurate approximation of the distribution of the test statistic. The bootstrap is a method to derive properties (standard errors, conïňĄ-dence intervals and critical values) of the sampling distribution of estimators. It takes the sample (the values of the independent and dependent random variables) as the population and the estimates of the sample as true values. Not to draw from a specified distribution by a random number generator, the bootstrap draws with replacement from the sample. The article will discuss many bootstrap methods and do simulations for some of them. 1 Definition 1.1 General illustration in Bootstrap World Consider a sample with n = 1, ..., N independent observations of a dependent variable y and M + 1 explanatory variables x. A paired bootstrap is obtained by independently drawing N pairs (xi, yi) from the observed sample withreplacement. The bootstrap sample has the same number of observations, however some observations appear several times and other observations never. The bootstrap involves drawing a large number B of bootstrap samples. a single bootstrap sample is denoted (x∗ b , y ∗ b), where x ∗ b is a N ∗ (M + 1) matrix and y ∗ b an N-dimensional column vector of the data in the b-th bootstrap sample. 1.2 Bootstrap Standard Errors The empirical standard deviation in a series of bootstrap replications of θ̂ can approximate the standard error se(θ̂) of an estimator θ̂. Here is the approach: • 1. Draw B independent bootstrap samples (x∗ b , y ∗ b) of size N from (x, y). Usually B = 100 replications are sufficient. • 2. Estimate the parameter θ of interest for each bootstrap sample: θ̂∗ b for b = 1, ..., B. • 3. Estimate se(θ̂) by ŝe = √ 1 B B ∑ b=1 (θ̂∗ b − θ̂ ∗)